Jeffrey Friedl's Blog
松中ジェフリーのブログ
文書は英語ですが、写真は共通
(though I'm taking a break from blogging for a while)
Simple JSON Encode/Decode in Pure Lua
August 1st, 2010 1:56am JST (15 years ago)
More Lua code

I've coded up some simple JSON encode/decode routines in pure Lua and thought I'd share them in case anyone else would find them useful. I use them in Adobe Lightroom, but they're pure Lua 5, so can be used anywhere Lua is.

JSON Encode/Decode in Pure LUA by Jeffrey Friedl is licensed under a Creative Commons Attribution 3.0 Unported License.
Download JSON.lua
Version 20211016.28 (version 28: October 16, 2021)

Full docs and changelog are in the code itself, but basic use is:

    JSON = (loadfile "JSON.lua")() -- one-time load of the routines

    local lua_value = JSON:decode(raw_json_text) -- decode example

    local raw_json_text    = JSON:encode(lua_table_or_value)        -- encode example
    local pretty_json_text = JSON:encode_pretty(lua_table_or_value) -- "pretty printed" version

Enjoy.

(You might also be interested in this comparison of Lua JSON packages.)

You can submit a comment below | Comments via RSS | Trackback
The 30 most-recent comments (out of 45; see all), most recent last...

HI Jeffrey,
When decoding a 900K json file, json lib will cost 14M+ memory. I review json.lua, and find use many stringcat .. operation at grok_string function.
so I change VALUE = VALUE .. c to tmp_stream:insert(c). diff code is here: https://gist.github.com/lvzixun/80e5b900b82059ebf5d7/revisions. using table.concat would be faster and reduce string memory allocation.

you can review code : https://gist.github.com/lvzixun/80e5b900b82059ebf5d7 😉

Unfortunately, whatever this “insert” function is, it’s not available in the version of Lua that I use. —Jeffrey

— comment by zixun on June 12th, 2015 at 12:55pm JST (10 years, 3 months ago) — comment permalink

Forked this repo for LuaRocks. Do you have GitHub account for this?
https://github.com/jiyinyiyong/json-lua

No, sorry, no GitHub. —Jeffrey

— comment by jiyinyiyong on June 26th, 2015 at 11:02pm JST (10 years, 2 months ago) — comment permalink

Hi Jeffrey,
thanks for the code. This is exactly what I was looking for. Works like a charme – in a Lightroom plugin!
So far, only simple JSON responses are decoded (did this with a regexp before), but it will probably become more important when using more sophisticated WebAPIs of the Synology PhotoStation.

Greetings from Berlin,

Martin

— comment by Martin on September 19th, 2015 at 7:35am JST (9 years, 11 months ago) — comment permalink

Hi Jeffrey,
Thanks a lot for this code ! Excellent work ! We are using this serilizer together with Zabbix JSON-RPC API in a large distributed monitoring network. This code provided us with an excellent jumpstart at first when we made a prototype of the gateway from Zabbix to internal ESB. After a while we have decided not to touch anything as it works perfectly and performance is sufficient for this project.

Thanks a lot again,
Greetings from Kazakhstan,
Pavel.

— comment by Paul on October 7th, 2015 at 7:18pm JST (9 years, 11 months ago) — comment permalink

Hello

I am new to Lightroom plugin development: I can’t find the correct syntax to load your JSON.lua file. I have put the JSON.lua file in the plugin folder. I have tried some code found on this page:
JSON = (loadfile “JSON.lua”)()
JSON = assert(loadfile “JSON.lua”)()
JSON = loadfile(LrPathUtils.child(_PLUGIN.path, “JSON.lua”))
without success.
Could you please give the correct syntax to use (Lightroom 6 Plugin SDK ).
Thanks a lot
Thierry

In Lightroom you can use `JSON = require "JSON.lua"`, but note that Lightroom won’t recognize any file added to a plugin folder after it starts, so after having added the JSON.lua file, you have to restart Lightroom before anything will work. —Jeffrey

— comment by Thierry on October 26th, 2015 at 10:21pm JST (9 years, 10 months ago) — comment permalink

During encode(), numbers larger than 1e14 were being changed to scientific notation.

Here is some lua code to show how tostring() handles large numbers
> x=100000000000001
> print(tonumber(x));
1e+14
> print(string.format(“%.0f”,x))
100000000000001

I changes the encoding to use the “%.0f” format -> works great

Here is the GIT DIFF
@@ -837,7 +837,10 @@ function encode_value(self, value, parents, etc, options, indent)
—
return “-1e+9999”
else
– return tostring(value)
+ return string.format(“%.0f”, value);
end

Ah, good catch, but unfortunately it’s not quite that simple. Try with your `x` value set to 10000000000054321, 10000000000054322, and 10000000000054321+1 and you’ll see (if your system is the same as mine) that at some point the string.format() version is wrong, due, I’m sure, to precision being lost in encoding big numbers in too-small a number of bits. I’d rather see that loss of precision represented, instead of a wrong number presented precisely.

There’s a range of decimals for which string.format() is indeed better, as your example shows; if I can determine for sure such cases, I’ll incorporate a change for them. I just tried the obvious way and it was lacking, but I’ll dig further.

Of course, we still need to make sure that a number like 3.14159 is output correctly.—Jeffrey

— comment by Russell Sullivan on April 22nd, 2016 at 9:29am JST (9 years, 4 months ago) — comment permalink

I am not able to do loadfile with LUA called from HAProxy config

JSON = (loadfile “JSON.lua”)()

The program exists for loadfile call.
The caller lua and JSON.lua are placed in same folder.

Is there specific configuration needed for HAProxy?

— comment by Deepak Agarwal on May 17th, 2016 at 2:19am JST (9 years, 3 months ago) — comment permalink

Any thoughts or ways around having decode properly handling nil values from a table?

For instance:
{
“somevalue”: null
}

will create a empty output since lua doesn’t support nil values as a key in a table element.

I’ve locally (and horribly) solved this by patching if nil to return a function that returns nil — but looking at the table I have to account for that too — then on the encode I just say if it’s a function then it is nil — good enough for me, but not for a general stance.
local function grok_object(self, text, start, etc)
[…]
VALUE[key] = new_val or function() return nil end

function encode_value(self, value, parents, etc, options, indent)
if type(value) == ‘function’ then value = nil end
if value == nil then
return ‘null’

Other than this small hiccup, I really love how the rest works… Efficient and stable!

As of version 20160526.15 I’ve added a “null” field to the encoding-options table. If your Lua table has a string value with the same value as the “null” option, it gets encoded into JSON as a raw null. Using `null="\0"` might be appropriate, for example. —Jeffrey

— comment by Adam B on May 25th, 2016 at 5:50pm JST (9 years, 3 months ago) — comment permalink

Thanks a lot for this! I’m using to create and load save files for a game I’m working on, and it works great. One small suggestion would be to ignore functions while parsing, rather than throwing an error. This is only my opinion, of course.

Anyway, I am quite happy with this library. It was the first result when I searched for “lua json parser.”

As an aside, what do you use this library for in Lightroom?

Since you asked where I’m from: Portland, Oregon.

I’m not sure what you mean by “ignore functions while parsing”; JSON has no concept of “functions”. I use this in my Lightroom plugins to handle responses from a variety of photo-hosting-site’s APIs. —Jeffrey

— comment by MarkSill on May 29th, 2016 at 3:58pm JST (9 years, 3 months ago) — comment permalink

I have found a bug when decoding an empty object, it is interpreted as an empty array, at least that is what shows on encode().
look at the field: ‘attributes’, it is an object in the string and then an array on decode()

lua
> JSON = (loadfile “./JSON.lua”)();– one-time load routines
> x='{“id”:”UserInfo_BILL”,”actions”:[],”attributes”:{}}’
> y=JSON:decode(x)
> print(JSON:encode(y));
{“actions”:[],”attributes”:[],”id”:”UserInfo_BILL”}

As mentioned in the docs, it’s handled correctly if you “`JSON.strictTypes = true`“. It’s not the default because supporting it requires adding metatables to the data, which can break some systems into which Lua is embedded. —Jeffrey

— comment by Russell Sullivan on June 4th, 2016 at 5:42am JST (9 years, 3 months ago) — comment permalink

Thank you for making this module!

I think I have spotted an unintended global. On line 516, you have:
> local isNumber = {
> __index = isNumber,
> — <snip>
> }
However, when __index = isNumber on line 517 is parsed, nothing has been assigned to the local isNumber variable on line 516 yet, so at that point isNumber is treated as an uninitialised global, and __index is assigned a value of nil. (At least, this is how it works in Lua 5.1.)

I haven’t followed all the paths through the code to see if this causes any problems, but I guess that this isn’t what was intended.

Holy cow, you’re right. What a nubie mistake on my part. Good catch, thanks. Fixed. —Jeffrey

— comment by Jack Taylor on September 16th, 2016 at 3:22pm JST (8 years, 11 months ago) — comment permalink

Hi!
This is an awesome module, thanks a lot!

Wanted to know, sadly in the project I’m testing, ‘the order’ of some keys in the json file needs to be preserved, my question is, is it possible when reading a json to keep the original order in which they were read?
decoding/encoding the same file changes the order of key-value pairs.

It’s not currently possible, sorry. Lua tables have no order for non-numeric keys, so to preserve the order you’d have to use metatables somehow, which just opens up a can of worms for handling that metadata (e.g. what to do when new keys are added in Lua). —Jeffrey

— comment by Alejandro on September 29th, 2016 at 7:44am JST (8 years, 11 months ago) — comment permalink

Hello again, and thank you for the updates to the module dealing with trailing garbage. I’ve spotted another nil global bug in the new code, so I thought I would let you know.

On line 1065, you have this variable declaration:

> local error_message

This declaration is in the scope of the if block starting at line 1060.

> if next_i ~= #text + 1 then

The problem is that on line 1070 you try and use error_message:

> return value, error_message

But at that point we have left the scope of the if block, so error_message is a nil global.

If you could update the code to fix this, it would be much appreciated.

Oops, sorry, I fixed this earlier in the month but goofed up pushing out the fixed version here. It (version 20161109.21) is here now. —Jeffrey

— comment by Jack Taylor on November 24th, 2016 at 12:29pm JST (8 years, 9 months ago) — comment permalink

Hey, do you have the old versions 1-20 somewhere online?
It would be cool to have a look at them (push them to git for proper version history/changes etc).

Cheers

You can append the version string to the download link to get old ones, e.g. `http://regex.info/code/JSON.lua-20140920.13` —Jeffrey

— comment by Gregor on March 17th, 2017 at 7:32pm JST (8 years, 5 months ago) — comment permalink

Wow Jeffrey,

Thank you a million times for this library.

Jon

— comment by Jon on April 4th, 2017 at 7:30am JST (8 years, 5 months ago) — comment permalink

local stock= ‘{“open”:332.44,”close”:329.87,”high”:332.5,”low”:328.15,”volume”:11038600.0,”actual_close”:331.29}’
local s = JSON:decode(stock)
return s[“high”]

This gives me a result of 332
or if I add JSON.decodeNumbersAsObjects = true I get: (empty list or set)

What do I need to get the right answer, 332.5?

Are you sure you’re not using it in some kind of integer context once you get the value back from whatever function this is? I changed your ‘return’ line to a ‘print’ line, and I got the proper 332.5. —Jeffrey

— comment by Greg on April 5th, 2017 at 12:57am JST (8 years, 5 months ago) — comment permalink

You’re right, it turns out Redis turns floats into integers. Not sure about the behavior of JSON.decodeNumbersAsObjects = true.

Thanks for the library, I’m new to lua so still asking stupid questions.

— comment by Greg on April 13th, 2017 at 9:07pm JST (8 years, 4 months ago) — comment permalink

hi, Jeffrey

I add a ‘array_newline’ option, which show array
`
[11,22,[33,44]]
`

as
`
[
| 11,
| 22,
| [
| | 33,
| | 44
| ]
]
`

https://github.com/yurenchen000/json-lua/commit/e870c6e702deacd3b9c5c2b1135893791fc51e16

thank you~

Seems like a good idea… I’ve added it, thanks! —Jeffrey

— comment by yurenchen on April 16th, 2017 at 4:59am JST (8 years, 4 months ago) — comment permalink

Hi Jeffrey,
Thanks for your module! I am running into a problem and I cannot figure out how to solve it. I would appreciate some help, please.

I have a json object being returned from a server, which seems to be properly decoded into a LUA table as I can reference it’s properties. One of the properties contains another json like string and I am trying to convert it to a table but keep getting “JSON.decode must be called in method format.” Here is the complete JSON string:

{“Success”:true,”GalleryId”:20094,”postURL”:”https://upload.blah.net/upload/processUpload/?websiteId=5140\u0026eventId=6500″,”FileUploadId”:”b5ymbczi”,”UMIs”:”{\”IMG_3271.JPG\”:\”109678\”,\”IMG_3272.JPG\”:\”109679\”,\”IMG_3273.JPG\”:\”109680\”,\”IMG_3274.JPG\”:\”109681\”}”}

I can isolate the last property called response.UMIs and print it out as the following string.
2017-06-05 18:59:19 +0000, TRACE UMIs as string: {“IMG_3271.JPG”:”109678″,”IMG_3272.JPG”:”109679″,”IMG_3273.JPG”:”109680″,”IMG_3274.JPG”:”109681″}

UMIS = {}
When I try UMIs = JSON.decode(response.UMIs) is when I get the error.
The JSON string is verified by a JSON linter so I am not sure what I am doing wrong.

Thanks!

Use “`JSON:decode`” rather than “`JSON.decode`” and it should work. —Jeffrey

— comment by Julian Dormon on June 6th, 2017 at 4:09am JST (8 years, 3 months ago) — comment permalink

Thanks man, this is exactly what I needed. I was just messing around with modding a game that I am playing at the moment. I was looking to export some data as JSON, so that I persist state between gaming sessions. This did the trick perfectly. You’ve saved me a bunch of time, and I really appreciate that you make this free to use.

Thank you!

— comment by Francois on July 2nd, 2017 at 2:30pm JST (8 years, 2 months ago) — comment permalink

I’d like to request support for boolean keys, right now it says “can’t encode table with a key of type boolean”. A simple fix is to have

…
if type(key) == ‘boolean’ then
table.insert(string_keys, tostring(key))
elseif type(key) == ‘string’ then
…

Makes sense. Done. —Jeffrey

— comment by Anonymous on August 19th, 2017 at 9:12am JST (8 years ago) — comment permalink

Using VERSION = ‘20170416.23’

At line 1325:

local encode_value – – must predeclare because it calls itself
function encode_value

Actually, you don’t need to predeclare a recursive function in Lua. The interpreter includes the function in its own _ENV before calling the function. You only need to predeclare pairs of recursive functions, like this:

local a
local function b()
a()
end
function a()
b()
end

I know about that property of Lua and use it elsewhere in the file, so I suppose there was something esoteric happening at one point during development that made me feel the need to do it explicitly. The Lua docs are pretty clear that the two expressions are identical, so I’m at a loss to guess what it might have been, but in any case, you’re right and I’ve tidied it up. Thanks. —Jeffrey

— comment by Chronos Phaenon Eosphoros on August 21st, 2017 at 11:26pm JST (8 years ago) — comment permalink

Using VERSION = ‘20170416.23’

I’m using you package for debugging code, so I need to encode table which includes functions. For my purposes, I just added this to encode_value():

elseif type(value) == ‘function’ then
return tostring(value)

However, as a suggestion, I think it would be nice if you either:

1- included some kind of types whitelist as an option for encoodin; or
2- made JSON:onEncodeError() interactive, as is JSON:onDecodeError(); or
3- both of the above.

Anyways, I’m quite happy with this package. Very nice implementation, quick, efficient and very useful.

Good idea, thanks. Check out `unsupportedTypeEncoder()` in the new version I just posted. —Jeffrey

— comment by Chronos Phaenon Eosphoros on August 21st, 2017 at 11:34pm JST (8 years ago) — comment permalink

Hi Jeffrey,

Great package! We’ve been using it for a while now and it works pretty well. This seems to have been mentioned before but as of Version 20170823.25 (version 25: August 23, 2017) we haven’t been able to pass a null option for *decoding* JSON strings. This is useful to us because we need to retain keys in the lua table (since there is no distinction between keys that don’t exist and keys assigned as nil values). Having null configurable when encoding a JSON string is not sufficient because we decode, modify, and then subsequently encode it again. The patch we made was as follows:

`
1086: - return nil, start + 4
1086: + return options.null, start + 4
`

Is there any chance that this patch could be surfaced for the next release?

Cheers,
Max

Oops, yeah, that was mentioned four years ago… I completely missed it, sorry. Thanks for the bump. I’ve just pushed the update. —Jeffrey

— comment by Max Sindwani on September 27th, 2017 at 6:28am JST (7 years, 11 months ago) — comment permalink

I’m trying to call JSON:decode() from inside an a startAsyncTask() call. From time to time the string passed might not contain valid JSON and decode() will raise an error. Lightroom, though, buries errors inside tasks which means I have no way to catch the error. Checking the string for valid JSON before passing to decode isn’t trivial. I wondered if you had anything to suggest.

many thanks.

You can trap such errors by using `LrFunctionContext.postAsyncTaskWithContext()` and immediately adding a cleanup or failure handler to the context passed to the new task. —Jeffrey

— comment by kim on September 28th, 2018 at 1:24am JST (6 years, 11 months ago) — comment permalink

The encode/decode process is causing loss of precision for floating point numbers. I changed the tostring() call in encode _value to string.format(‘%20.16g’,value) to fix the problem.

Could you give an example where the original code was losing precision? —Jeffrey

— comment by Don on September 3rd, 2019 at 11:59pm JST (6 years ago) — comment permalink

Thank you very much for perfectly solving the problem of loss of precision caused by big number deserialization; Nginx performance loss is also small. Example of use:

JSON = require(“JSON”)
JSON.decodeNumbersAsObjects = true
lua_value = JSON:decode(‘{“num”:10201348204923842304}’)
raw_json_text = JSON:encode(lua_value)
print(raw_json_text)

— comment by fangjianfeng on September 6th, 2019 at 12:13pm JST (6 years ago) — comment permalink

It is amazing work. Thank you for the sharing!

— comment by Kevin Q on January 31st, 2020 at 4:06am JST (5 years, 7 months ago) — comment permalink

Hi Jeffrey,

I will report an error when parsing. I think utf8 coding is not supported

“[string “chunk”]:688: [string “chunk”]:194: incomplete utf8 sequence at byte 16 of: {“materielid”:”塑胶原”,”materielcode”:”610.43420″}”

PS: I’m from China.

It works for me when I try your example, but it’s possible that the encoding was changed by WordPress. Could you email a ZIP file of some sample data that you encounter the error with? `jfriedl@yahoo.com` Thanks. —Jeffrey

— comment by jack on October 19th, 2021 at 11:26am JST (3 years, 10 months ago) — comment permalink

Hi Jeffrey,

Great library, thanks for sharing. I’m curious is there a way to make encoding happen in the order I create a table in Lua so keys are not sorted in alphabetic order? Let say I have a table t = {z = “some value for z”, a = “some value for a”} if I’ll encode it I’m going to get {“a”:”some value for a”, “z”: “some value for z”}

Thank you.

No, I don’t think that’s possible. As far as I know, there is no temporal information available in Lua tables to indicate any kind of added/modified order. —Jeffrey

— comment by Ilya on October 28th, 2021 at 11:25pm JST (3 years, 10 months ago) — comment permalink
Leave a comment...

Name

|  | Email address (optional; will not be published) Notify me of followup comments, or when Jeffrey replies, via email. Fill in your email and check this box if you expect a reply from Jeffrey and don't want to keep checking the comments manually.

Website (if you have one and would like to share it)

Enter “Happy” (without quotes) in this box (required — this helps stop spambots)

All comments are invisible to others until Jeffrey approves them.

Please mention what part of the world you're writing from, if you don't mind. It's always interesting to see where people are visiting from.

IMPORTANT: | I'm mostly retired, so I don't check comments often anymore, sorry.

You can use basic HTML; be sure to close tags properly.

Subscribe without commenting
E-Mail: